Method and apparatus for tracking liquid consumption behavior

ABSTRACT

A method of detecting, measuring, logging, and tracking a user&#39;s consumption of liquid through an electronic wearable on a person&#39;s wrist, hand, or other extremity used for consuming beverages is described. In preferred embodiments, the electronic wearable can detect individual sips, the final sip of a beverage, and the type of container the beverage is being consumed from.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention is directed to a method of detecting, measuring,logging, and tracking a user's consumption of liquid through anelectronic wearable on a person's wrist, hand, or other extremity usedfor consuming beverages.

Description of the Related Art

Logging consumption of food or liquids has been a common practice forathletes attempting to maximize training and performance results, aswell as non-athletes on a diet trying to either live a healthierlifestyle or lose weight. With the invention of smartphones, tablets,and laptops, logging food and liquid consumption have become easier withsoftware dedicated to these specific tasks. Regarding fluidspecifically, existing phone apps help users log their fluid intake to,for example, ensure proper hydration, monitor estimated blood alcohollevels, and avoid ingesting too much caffeine.

A new generation of wearable electronic devices now allows even moreconvenient means of tracking dietary and fitness levels. Certainwearable electronic fitness trackers advertise dietary logging includingfood, water, and caffeine. However, these devices require the user toinput information (e.g., type and amount of beverage), rather thanhaving that information tracked automatically through recognition ofconsumption by the device itself. The problem with prior art trackingmechanisms is their inherent need for the user to manually enter thetype of drink, the amount consumed, and the time either when theconsumption is taking place or at a later time with a time stamp back towhen the consumption took place, which ultimately leads tonon-compliance with dietary or athletic performance goals.

U.S. patent application US 2014/0372045, for example, attempts toresolve the problem of requiring manual input of liquid consumption byteaching an electronic band that attaches to a fluid container andrecords drinking events through the sensors embedded in the band. Thisinformation and data collected from the band is relayed to a smartphoneor tablet via wireless communication means, where the information isanalyzed by a processor. While this invention eliminates the need forthe user to enter each sip of water or other type of drink consumed, itinconveniently requires the user to carry around a band and sensorspecifically to place on each container the user drinks from wheneverand wherever the user drinks throughout the course of days or weeks toachieve desired fitness or dietary results. That invention also requiresthe need for the user to enter information about the container the useris consuming a beverage from, which, as with any constant need for dataentry, can be burdensome.

Thus, the need exists for a wearable electronic device that can detectthe consumption of a beverage by a user with less, limited, or no manualinput from said user into an application or physical notebook. Thewearable electronic device and/or method of tracking drinking behaviorbecome more useful if they can detect the type of fluid container thebeverage is being consumed from, and ultimately detect the type of fluidbeing consumed. This eliminates the need for the user to keep track ofbeverage consumption by memory, manual logging, or digital logging.

SUMMARY OF THE INVENTION

An object of this invention is to detect and monitor liquid consumptionthrough the use of a wearable electronic device, without the need for anelectronic device to be placed around the drinking apparatus, or forburdensome data entry. The results of information and data received fromeach consumption process and multiple consumption processes over timecan be analyzed by a program or programs on the wearable device or someother electronic device linked to the wearable device to trackconsumption, extrapolate results to the dietary or athletic goals of auser, or otherwise analyze the drinking patterns of a user and theresults thereof.

Another object of this invention is to detect and record what kind ofcontainer the user is drinking from in which the liquid is contained(e.g., a bottle, a can, a mug, etc.), which information may be used inthe downstream analysis of the results of a user's drinking behavior.

Another object of this invention is to detect and isolate a consumptionprocess based on hand gestures compared to other hand movements,allowing tracking of drinking as opposed to other tasks, (e.g., writing,eating, and other tasks involving the hands), in order to both monitorfluid intake and determine certain properties of the liquid's container.

Another object of this invention is to provide nutritional informationto the user based on the amount and type of liquid consumed, includingthe amount and type of liquid consumed over set periods of time (e.g.,one day, one week, one month, one year, and so on).

In one embodiment of the present invention, an arm, hand, or wristbracelet or band with electronic features (e.g., a wearable device) willbe worn around the wrist of a user. This wearable should have at leastone sensor capable of measuring accelerations, motions, orposition/attitude of the wrist and/or hand. A microprocessor receivesinformation collected by the sensor(s) and processes that information.In a preferred embodiment, the wearable contains memory storagecapability to store the processed data collected by the sensor(s). Thewearable should have a means of communicating with a phone, tablet,computer, other peripheral device, or the internet. This communicationcan be achieved through a radio frequency (rf) signal like Bluetooth orbut this can also be done through a physical cable such as a USB cable.The preferred embodiment will use Bluetooth to communicate with asmartphone and software on said smartphone. The wearable can alsoinclude a display to communicate necessary information to the user or atouch screen, similar to a smart watch (e.g., an Apple Watch®) to allowthe user to both see and input information. The wearable can alsocontain multiple physical or digital buttons to input information.

In preferred embodiments, the microprocessor will have an algorithmprogrammed into it that will be able to recognize a prescribed set ofmotions through accelerometer(s) in the wearable which will detect andisolate the consumption process from all other hand movements andgestures. This desired result can also be achieved with a gyroscope inthe wearable device. Once the event is recognized, the microprocessorwill be able to save important data to the memory on the wearabledevice, or externally. By analyzing the data of the consumptionprocesses, the amount of liquid, type and/or size of the container, andother information, will be determined. When combined with informationabout the type of liquid consumed, as processed by the microprocessor orother processing means, the user will be able to see nutritionalinformation about what they have consumed such as amount of water,caffeine, sugar, alcohol, and other figures. This will allow the user totrack fluid intake, and use that data to compare against dietary and/orathletic goals, or to maximize drinking behavior to result in improveddietary, physical, and/or mental health.

Specific aspects of the current invention include Aspect 1, which is amethod of determining whether a user is drinking liquid, the methodcomprising: measuring changes to the x, y, or z axes using anaccelerometer or gyroscope located on a wearable electronic device;measuring changes to the total g force using an accelerometer on saidwearable electronic device; having a microprocessor analyze data from x,y, or z axes along with total g forces to determine if the changesindicate the user of the wearable electronic device is drinking.

Aspect 2 is the method of Aspect 1 wherein one or more of the followingsteps are detected to thereby indicate whether the user is drinking:

-   a. Reaching for drink;-   b. Grabbing drink;-   c. Lifting drink to mouth;-   d. Tilting until liquid touches lips;-   e. Tilting while consuming;-   f. “Unfitting” (or reversing the drinking tilt) to normal holding    position;-   g. Lowering drink; and-   h. Returning drink and arm.

Aspect 3 is a method of determining whether a wearer of an electronicdevice having an accelerometer or gyroscope is drinking based on changesto the x, y, and/or z axes and total g forces indicates the wearer isreaching for a drink.

Aspect 4 is a method of determining if a wearer of an electronic devicehaving an accelerometer or gyroscope is drinking based on whetherchanges to the x, y, and/or z axes and total g forces indicate thewearer is grabbing a drink.

Aspect 5 is a method of determining if a wearer of an electronic devicehaving an accelerometer or gyroscope is drinking based on whetherchanges to the x, y, and/or z axes and total g forces indicate thewearer is lifting the drink to the wearer's mouth.

Aspect 6 is a method of determining if a wearer of an electronic devicehaving an accelerometer or gyroscope is drinking based on whetherchanges to the x, y, and/or z axes and total g forces indicate thewearer is tilting the drink until the liquid touches the wearer's lips.

Aspect 7 is a method of determining if a wearer of an electronic devicehaving an accelerometer or gyroscope is drinking based on whetherchanges to the x, y, and/or z axes and total g forces indicate thewearer is tilting the drink to consume it.

Aspect 8 is a method of determining if a wearer of an electronic devicehaving an accelerometer or gyroscope is drinking based on whetherchanges to the x, y, and/or z axes and total g forces indicate thewearer is untilting the drink back to a normal holding position.

Aspect 9 is a method of determining if a wearer of an electronic devicehaving an accelerometer or gyroscope is drinking based on whetherchanges to the x, y, and/or z axes and total g forces indicate thewearer is lowering the drink.

Aspect 10 is a method of determining if a wearer of an electronic devicehaving an accelerometer or gyroscope is drinking based on whetherchanges to the x, y, and/or z axes and total g forces indicate thewearer is returning the drink to a resting position.

Aspect 11 is a method of determining whether a wearer of an electronicdevice having an accelerometer or gyroscope is drinking based on changesto the x, y, and/or z axes and total g forces, as measured by theaccelerometer or gyroscope, indicating the wearer is performing two ormore of the following actions:

-   1. Reaching for drink;-   2. Grabbing drink;-   3. Lifting drink to mouth;-   4. Tilting until liquid touches lips;-   5. Tilting while consuming;-   6. “Untilting” (or reversing the drinking tilt) to normal holding    position;-   7. Lowering drink; and-   8. Returning drink and arm.

Aspect 12 is a method of determining whether a wearer of an electronicdevice having an accelerometer or gyroscope has taken the last sip of abeverage from a drinking container based on measured changes to the x,y, and/or z axes using said accelerometer or gyroscope.

Aspect 13 is a method of determining whether a wearer of an electronicdevice having an accelerometer or gyroscope has taken the last sip of abeverage from a drinking container based on measured changes to the x,y, and/or z axes along with measured total g forces using saidaccelerometer or gyroscope.

Aspect 14 is a method of tracking drinking behavior comprising: using awearable electronic device to detect changes in the x, y, or z axis,along with changes in g forces; and using a microprocessor to analyzethe measured drinking behavior.

Aspect 15 is a method of tracking drinking behavior comprising: using awearable electronic device to detect changes in the x, y, or z axis,along with changes in g forces; using a microprocessor to analyze themeasured drinking behavior; and outputting to the user the results ofthe analysis.

Aspect 16 is a method of tracking drinking behavior comprising: using awearable electronic device to detect changes in the x, y, or z axis,along with changes in g forces; using a microprocessor to analyze themeasured drinking behavior; and outputting to the user consequences ofthe drinking behavior based on metrics related to physical performance,as well as physical and mental health issues impacted by drinkingbehavior.

Aspect 17 is a method of determining the type of container from which awearer of a wearable electronic device is drinking comprising: loggingthe peak x axis accelerations during the consumption phase; and runninga regression on said peak x axis acceleration points in order toclassify the type of container based on set parameters and otherprevious data from user.

Aspect 18 is a method of determining the type of container from anelectronic device placed on a beverage container with a sensor orsensors being consumed comprising: logging the peak x axis accelerationsduring the consumption phase; and running a regression on said peak xaxis acceleration points in order to classify the type of containerbased on set parameters and other previous data from user.

Aspect 19 is a method of determining whether a person is drinking abeverage by using an electronic device placed on a beverage containerhaving a gyroscope based on measured changes to the x, y, and/or z axesusing said gyroscope.

Aspect 20 is a method of determining Whether a person has consumed thelast sip of a beverage by using an electronic device placed on thebeverage container having an accelerometer or gyroscope based onmeasured changes to the x, y, and/or z axes, and by comparingmeasurements to previous peak accelerations seen during the consumptionphase.

Aspect 21 is a method for determining whether a user is consuming afluid through use of an electronic device capable of being attached toor worn on an extremity of a user, comprising:

A first motion sensor capable of detecting movement, position, and/ororientation located within a housing for the portable electronic device,and

A microprocessor capable of receiving, interpreting, and analyzingsignals from said motion sensor and able to run programmed algorithms,and

An algorithm capable of detecting when said user consumes a liquid byrecognizing specific gestures (movements and change in position and/ororientation) and capable to filtering out other non-drinking movements.

Aspect 22 is a method of Aspect 21, wherein the motion sensor is anaccelerometer and the method of determining whether a user is consumingfluid is by comparing total accelerations to the individualaccelerations of axes and/or position/orientation of said user'swrist/hand.

Aspect 23 is a method of Aspect 21, wherein the motion sensor is agyroscope and the method of determining whether a user is consumingfluid is by analyzing the specific orientation and position of saiduser's hand/arm/extremity.

Aspect 24 is a method of Aspect 21, wherein the detection algorithmdetects the consumption portion of the consumption process by comparingoverall movement of user's extremity to changes in position and/ororientation of said extremity.

Aspect 25 is a method of Aspect 24, wherein a change in position and/ororientation is determined by monitoring the pitch angle (rotation aboutthe axis parallel to said user's forearm).

Aspect 26 is a method of Aspect 24, wherein a change in position and/ororientation is determined by monitoring the x axis acceleration and thetotal acceleration of said extremity

Aspect 27 is a method of Aspect 24, wherein a change in position and/ororientation is determined by using a gyroscope.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings illustrate certain aspects of embodiments ofthe present invention, and should not be used to limit the invention.Together with the written description the drawings serve to explaincertain principles of the invention. A wide variety of potentialembodiments will be more readily understood through the followingdetailed description of certain exemplary embodiments, with reference tothe accompanying exemplary drawings in which:

FIG. 1 shows an example of a preferred embodiment of the wearableelectronic device.

FIG. 2 shows the wearable electronic device of FIG. 1 being worn on awrist of a user in a preferred location to track consumption processes.

FIG. 3 illustrates the preferred electrical components of an exemplarywearable electronic device.

FIG. 4 illustrates the electrical components of the wearable electronicdevice in its most simplistic state.

FIG. 5 illustrates the electrical components of the wearable electronicdevice with possible options.

FIG. 6 presents the accelerometer axis orientation and direction ofgravitational force for the experiments and data with a sensor mountedon the fluid container.

FIG. 7 presents the accelerometer axis orientation and the direction ofgravitational force for the experiments and data collected when thesensor is located on a wearable electronic device worn on a wrist.

FIG. 8A shows three axis accelerations and the total acceleration whenthe sensor is placed on the fluid container during the consumption ofone entire beverage.

FIG. 8B shows the x axis accelerations when the sensor is placed on thefluid container during the consumption of one entire beverage.

FIG. 9 shows three axis accelerations and total acceleration when thesensor is placed on the wrist of the user during the consumption of oneentire beverage.

FIG. 10 illustrates the flow of information between the wearableelectronic device, a phone/tablet/computer or other external computingdevice, and a cloud/server.

FIG. 11 is a flowchart representing what occurs when a consumptionprocess is detected or when a false or non-event is detected, and therelated processing information.

FIG. 12A is a flowchart showing steps in determining and detectingwhether a final sip has occurred.

FIG. 12B is a flowchart showing steps in determining and detectingwhether a final sip has occurred.

FIG. 13 is a flowchart showing steps in determining and detectingwhether a final sip has occurred.

FIG. 14 is a flowchart showing an embodiment of steps in determining anddetecting whether a final sip has occurred.

FIG. 15 illustrates eight gestures of drinking that are used, in wholeor in part, in an algorithm that detects and measures the user consuminga beverage, as detectable by a wearable electronic device worn on theextremity (e.g., the wrist) used to hold the fluid container.

FIG. 16 is a chart showing the accelerations from all three axes alongwith the total acceleration measured by a sensor worn on a user's wristduring the consumption of liquid from a liquid container.

FIG. 17 is the same chart shown in FIG. 16 isolating, in numberedvertical segments, each of the eight steps of consuming a beveragedepicted in FIG. 15.

FIG. 18 is a chart displaying the on wrist x axis accelerations duringthe consumption of one entire beverage, from a first sip S₁ to a lastsip S_(n).

FIG. 19 is a chart plotting the peak x axis acceleration from anon-drink sensor during each consumption process throughout theconsumption of a beverage. Varying types of liquid containers arecompared against each other, and trend lines show the linear regressionresults for each container type.

FIG. 20 is a chart plotting the peak x axis acceleration from anon-wrist sensor during each consumption process throughout theconsumption of a beverage. Varying types of liquid containers arecompared against each other, and trend lines show the linear regressionresults for each container type.

FIG. 21A is a flowchart illustrating a sip detection algorithm thatincludes all eight steps of the consumption process shown in FIG. 15.

FIG. 21B is a flowchart illustrating a simplified sip detectionalgorithm.

FIG. 21C is a flowchart illustrating a simplified sip detectionalgorithm that also includes the ability to determine the location ofthe fluid container.

FIG. 22 is a flowchart illustrating a final sip detection algorithmprimarily concerned with y and z axis accelerations during Step 5.

DETAILED DESCRIPTION OF VARIOUS EMBODIMENTS OF THE INVENTION

Reference will now be made in detail to various exemplary embodiments ofthe invention. It is to be understood that the following discussion ofexemplary embodiments is not intended as a limitation on the invention.Rather, the following discussion is provided to give the reader a moredetailed understanding of certain aspects and features of the invention.

FIG. 1 shows an example of a wearable electronic device, which, in apreferred embodiment includes an accelerometer for detecting movement,as well as processing capability to monitor, record, and analyzedrinking activity. The preferred location for the wearable electronicdevice is on the wrist (as pictured), but it could also be worn, forexample, on a finger, on a hand, or further up the arm if arrangeddifferently and able to detect drinking activity as disclosed herein.The exemplary wearable electronic device consists of a band to be wornaround the wrist of the user. This band can be constructed of anymaterial that will secure the wearable electronic device to the userincluding but not limited to rubber, leather, plastic, metal (aluminum,titanium, steal, etc.), neoprene, or any other material which would notinterfere with the invention disclosed in this patent application. Thehousing may, by way of example, contain the electronic portion of thewearable electronic device such as the power source, microprocessor,sensor, data storage, communication chip, etc., although these functionsmay be performed by electronics in the band or even external of thewearable device. The housing can be a separate portion that connects tothe band, or it can be contained inside of the band.

In a preferred embodiment, the wearable electronic device includes afirst sensor that comprises a three-axis accelerometer. The first sensorcould also comprise a gyroscope. With both of these sensors, three-axissensors are preferred, but could consist of a single axis sensor. Thewearable electronic device may also include a second sensor, which, inan embodiment, would comprise the combination of both an accelerometerand a gyroscope. When the term “motion sensor” is used herein, it isreferencing an accelerometer, gyroscope, or other sensor capable ofdetecting linear or rotational movements along or around at least oneaxis.

A preferred wearable electronic device will include a microprocessorwithin the device. The microprocessor will process the informationdetected by a motion sensor (e.g., accelerometer), interpret the data,connect with other devices (wired or wirelessly), and communicateprocessed results to the user directly or through another device,amongst many other functions. (See e.g., FIG. 4.) A microprocessor couldalso be outside the device but linked to the device for certain remoteprocessing (for example, physically or by means of Bluetooth, Wi-Fi, orsome other non-physical means).

A time stamp should be added to the motion sensor data received by theprocesser. Most microprocessors are capable of performing such function,but others may require the addition of an oscillator (e.g., quartz,oscillator) or some other means of keeping accurate time and adding atime stamp to the data. The term microprocessor and processor haverecently come to be used interchangeably. A processor could also be usedin place of a microprocessor in this invention.

In a preferred embodiment, the wearable electronic device will alsoinclude a means of storing data, although data could be storedelsewhere, such as on another device or in the cloud. In the preferredembodiment, the data storage will comprise two separate data storagedevices. One device will be used for temporary storage that will becontinually overwritten with new data, and the other data storage willbe used to store longer-term information regarding drinking behavioruntil that data can be sent to a phone, tablet, computer, or otherperipheral device. However, the invention described herein may beperformed with only one storage location, such as in the wearableelectronic device, within the cloud, or on a peripheral device. (See,e.g., FIG. 3 and FIG. 4.)

In a preferred embodiment, the wearable electronic device will alsoinclude a means of communicating with a phone, tablet, computer,peripheral device, and/or a server. This function can be achievedthrough a wireless connection such as Wi-Fi or Bluetooth, but can alsobe a physical connection, including, but not limited to, a USB cable ora ⅛″ headphone jack. In its preferred embodiment, the means ofcommunication will be performed using a Bluetooth connection. FIG. 2shows a preferred embodiment of a wearable electronic device on thewrist of a user holding a drinking apparatus—in this case—a mug.

FIG. 10 shows the flow of information from the wearable electronicdevice, to a phone, tablet, computer, or other peripheral electronicdevice, and then from the phone, tablet, computer, or other peripheralelectronic device to, in the example, a cloud based server. This flow ofinformation could eventually circumvent the need for a phone, tablet,computer, or other peripheral device if the processing capability andstorage capacity of the wearable electronic device increases and/or isable to have an internet or other non-physical connection to a server.

The wearable electronic device will also need to be powered from somepower source including but not limited to a battery, solar panel, or anyother means for powering the device.

The wearable electronic device may also utilize a button or switch. Forexample, a single button could be used to turn on the wearableelectronic device and allow the user to input necessary information.This could be done by way of one or multiple taps, or holding the buttondown for a specified period of time. Information can also be entered onthe face of the electronic wearable device, or on a peripheral device(e.g., a phone or personal computer) that sends such information to theelectronic wearable device. The preferred embodiment of this wearableelectronic device will include one button for the user to inputinformation and one switch to power on the device. The preferredlocation for the switch is in the clasp, which will automatically turnthe wearable electronic device on when it is clasped around the user'swrist. This can be done by employing a magnetic clasp, but can also bedone through many alternative methods, such as physical means when theband is clasped. A magnetic clasp can be seen in FIG. 1 where when thetwo ends are attached by means of a magnetic force, a connection orcircuit is also completed telling the microprocessor to turn on thewearable device.

The wearable electronic device can also include an indicator light,screen, or display to communicate information to the user. If aninteractive touch screen is used, some of the user interface that wouldnormally be performed on a phone, tablet, computer, or other peripheralelectronic device can be performed on the interactive screen. This couldalso take the place of the button or switch as described. The preferredembodiment of the wearable electronic device includes an indicator light(e.g., LED light).

Consumption Detection

U.S. patent application US 2014/0372045 A1 describes an electronicdevice with sensors, which fits around a drink container to detect whena sip has occurred and calculate fluid intake by a user. When thesensory band is placed on the liquid container, thespecifically-disclosed algorithm detects drinks and sips from monitoringthe movement of the container. FIG. 8A shows acceleration data takenfrom a fluid container during the course of its entire consumption.Because the sensors are on the container, any time a movement isdetected it can be assumed that the drink is moving, or changingorientation. This results in a simpler algorithm to differentiate thedrinking event from non-drinking events, as opposed to when the sensorsare on the wrist as taught by the current invention. The data recordedis also less complex than the data processed according the presentinvention, because when the fluid container is tilted duringconsumption, the exact orientation of the fluid container is known basedon the alignment of axis with the container itself (See, e.g., FIG. 8Bshowing only the single x axis measurement.) From this single axis (thex axis shown in FIG. 8a ), the tilt of the fluid container can bedetermined. U.S. patent application 2014/0372045 teaches thiscalculation in FIG. 11 of that application, and references the followingequation:

${{tilt}\mspace{14mu}{angle}} = {{{arcos}\left( \frac{y^{\prime}}{gravity} \right)} = {{arcos}\left( \frac{y^{\prime}}{1} \right)}}$(See FIG. 6 for axes and gravity in relation to a fluid container.)However, a more accurate manner in which to determine the tilt angle ofthe fluid container is to first calculate the total accelerations (nowreferred to as total g below) detected by the fluid container. Thistotal g can be determined by taking the root mean square ofaccelerations seen on all three axes (the x, y, and z axes):total g=√{square root over (x ² +y ² +z ²)}(See FIG. 7 for axes and gravity in relation to an electronic device asworn on the wrist.) A more accurate tilt angle can then be calculated byfactoring in total g instead of gravity as in the equation disclosed inU.S. patent application 2014/0372045. This results in filtering out someaccelerations from movements of the fluid container and primarilymeasuring the gravitational force's effect on the x axis.

U.S. patent application 2014/0372045 detects a drinking event by settingan event threshold. This looks only at the tilt angle of the fluidcontainer (determined by the equation above) and time. Looking at FIG.8B, an event threshold −0.7 g, where any x axis acceleration greaterthan the threshold would log a drinking event, all of the drinkingevents would have been successfully accounted for. If this algorithmwere applied to the x axis acceleration data from FIG. 9 (data takenfrom an on-wrist sensor location during the consumption of a beverage),it would be unable to determine if any drinking events took place due tothe varied and more complicated data collected from the wrist of theuser.

When the sensor is worn on the wrist as opposed to being placed on thebeverage container as taught by U.S. patent application 2014/0372045,the information detected and monitored changes dramatically, making itmore complex and therefore more complicated to distinguish drinkingevents from other non-drinking movements. FIG. 9 shows the accelerationstaken from a user's wrist during consumption of a liquid. By aqualitative comparison between FIGS. 8 (on container accelerations) and9 (on wrist accelerations), the increased difficulty in detectingdrinking vs. non-drinking events is represented. The invention describedherein solves the problem, allowing for consumption to be accuratelydetected by a sensor or sensors in a wearable device.

Through extensive data analysis and scientific experimentation, it turnsout the consumption processes can be detected and isolated from othermovements by measuring accelerations on a user's wrist, hand, finger, orother extremity used to hold and consume liquid, as opposed to requiringthe sensor to be placed on the beverage container. In order toaccurately determine if a consumption process is taking place, the x, y,and z axes need to be monitored in addition to the total accelerationson the fluid container, which will be referred to as “total g.” Thetotal g can be calculated by taking the root mean square ofaccelerations seen in all three axes:total g=√{square root over (x ² +y ² +z ²)}All four of these accelerations are displayed by FIG. 16 during a singleconsumption process. In addition to the accelerations in all three axesand the total acceleration, the rotation around certain axes can be usedto determine consumption processes. For detecting the consumptionprocess, the most useful rotation to look at is around the y axis, whichis parallel to the user's forearm. This is referred to herein as“rotation pitch” and “pitch.” The actual angle of the wearable asmeasured around the user's forearm will be referred to as “pitch angle.”All of the rotations can be calculated using the “aerospace rotationsequence” method. To calculate the approximate pitch angle (in radians)around the y axis, the following equation could be used:

${pitch} = {{artan}\left( \frac{x}{\sqrt{y^{2} + z^{2}}} \right)}$This equation also takes into effect a rotation around the z axis, butbecause the primary rotation during a consumption process is around they axis, this equation is used as an appropriate approximation formeasuring the pitch.

By analyzing the data of a consumption process, the act of drinking canbe separated into eight discrete steps shown in FIG. 15:

-   1. Reaching for drink;-   2. Grabbing drink;-   3. Lifting drink to mouth;-   4. Tilting until liquid touches lips;-   5. Tilting while consuming;-   6. “Untilting” (or reversing the drinking tilt) to normal holding    position;-   7. Lowering drink; and-   8. Returning drink and arm.    Each of these steps can be detected and measured as information and    data as shown in FIGS. 16 and 17. (FIG. 17 shows the same data as in    FIG. 16, but each discrete drinking step as numbered in FIG. 15 is    overlaid by shaded vertical columns.) Each of the discrete drinking    steps and corresponding data from FIG. 17 are explained in detail    below:

STEP 1—Reaching for drink: When the user reaches for the fluidcontainer, two distinct motions occur. The user moves his hand and wristtoward the drink container, usually in the y axis, and the user tiltshis wrist in order to grab ahold of the fluid container. Both of theseactions can clearly be seen in in FIG. 17 at the vertical columnnumbered 1. The y axis acceleration has an immediate rise and then asubsequent drop that correlates with the total g measured. Thisindicates a movement of the wrist toward the fluid container. Secondly,the x axis acceleration moves to almost −1 g. This change in x axisacceleration shows the reorientation of the wrist to a more verticalposition (along the x axis) in order to grab the fluid container. Notethat this reorientation of the wrist an also be seen in the z axis.

STEP 2—Grabbing drink: The act of grabbing the drink can be determinedby relatively stable (or flat) readings in all axes (x, y, and z) andthe total g with the y axis acceleration close to zero and the x axisacceleration close to −1 g. This signifies that the aria has stoppedmoving in order to reorient to grab the drink container, but has not yetmoved the fluid container closer the mouth of the user. (See column 2 inFIG. 17.)

STEP 3—Lifting drink to mouth: The act of lifting the fluid container tothe mouth of the user can be identified primarily by a change in the yaxis acceleration measurement, as the forearm is being moved from closeto a horizontal position up to an angle that moves the fluid containercloser to the user's mouth. When the y axis acceleration levels off, thefluid container can be assumed to be positioned near the user's mouth.It is also important to recognize that the x axis and z axisaccelerations are also rising as the wrist is beginning to tilt thefluid container. The movement of the wrist and fluid container will bemeasured by a quick fluctuation in the total g to both greater than andless than 1 g (starting the movement and stopping the movement). Anapproximate range for this fluctuation will be between 0.9 g and 1.1 g.(See column 3 in FIG. 17).

STEP 4—Tilting until liquid touches lips: During this step, thecontainer is already positioned at the user's mouth, meaning the y axisacceleration is relatively steady as shown in column 4 of FIG. 17. The xaxis and z axis accelerations continue to climb as the user moves hiswrist and arm to allow the liquid to touch his lips. When liquid istouching the lips, the user changes his movement during consumption.This is shown in the data by a leveling off of the z axis and slight dipin the x axis as depicted in column 4 of FIG. 17. The user usually has aquick pause at this moment before starting the consumption phase. Thismovement and pause is seen in the total g with a slight dip and returnto 1 g. This marks when the liquid is touching the user's lips.

STEP 5—Tilting while consuming: This step marks the actual consumptionof the fluid. Most notably, the y axis and z axis accelerations remainsteady and relatively flat during this step, in addition to almost nochanges in total g. The only sip that sometimes sees a drop in both yand z axis accelerations is the last sip (and potentially second-to-lastsip). This is due to a change in the movement (raising the elbow higherthan normal) compared to all other periods of consumption. Depending onhow the user is holding the fluid container, the y and z accelerationscan be different, but they will be steady. During the movements beforeand after the consumption phase, the movements in the wrist will belogged by the total g, but during consumption, this is stable andmeasures almost 1 g, the gravitational force. During all of this, the xaxis acceleration continues to climb as liquid is being consumed andeventually levels off as the consumption of liquid slows and eventuallystops. (See column 5 of FIG. 17.)

STEP 6—“Untilting” to normal holding position: The “untilting” phasemarks the end of consumption and return of the hand, wrist, and fluidcontainer to a normal upright position. This can be seen in the data bya reduction in the x axis acceleration and the start of a reduction inthe z axis acceleration. During this phase, the y axis measurements aremostly constant. As the “untilting” motion begins, the total g willstart to change, registering motion compared to the relatively steadyconsumption step. (See column 6 of FIG. 17.)

STEP 7—Lowering drink: The lowering of the fluid container can bedetected by rapid decrease in the x axis and y axis accelerations as thewrist returns to the mostly vertical position holding the fluidcontainer in an upright manner and the forearm moves from an inclined toa mostly horizontal position. As the x Axis decreases to almost −1 g, ittends to stabilize and stay near −1 g as the drink is still being held.The z axis is relatively steady during this phase. (See column 7 of FIG.17.)

STEP 8—Returning drink and arm: This step is comprised of moving thefluid container to its final position, releasing the container, andmoving of the user's arm to a non-drinking position or natural state.This step can be determined by a sudden rise and fall in the y axisacceleration and a change in the x axis acceleration once the drink hasbeen released. Small changes in the total g register the movement ofthis step. The z axis is measuring a portion of the movement androtation of the forearm. (See column 8 of FIG. 17.)

It is noteworthy that moving the sensor location from one wrist or handto the other (for example moving it from a user's right wrist to hisleft wrist) will affect the y axis measurements. In FIG. 17, the y axisacceleration measurements move from about 0 g to 0.7 g during Step 3. Ifthe sensor were to be worn on the other hand or wrist, the y axismeasurements would show the same movement, but in the oppositedirection. Lifting of the forearm would move measurements from about 0 gto −0.7 g in this example.

While all of the aforementioned steps together make up the motions andmovements detectable by a normal or full consumption process, some ofthese steps might not occur if the way in which the user is consumingthe beverage changes. For example, these steps require the fluidcontainer to be sitting down on something such as a table or bar, beforethe process begins. The sensors and processor then register the movementto pick up, consume the liquid, and return the fluid container to aseated position. If the user decides to hold the fluid containeralongside his body between sips or the actual consumption of thebeverage in step 5, steps 1, 2, and 8 will be eliminated. If the userdecides to hold the fluid container at his lips in-between twoconsecutive sips, steps 1, 2, 3, 7, and 8 will be eliminated. Thealgorithm disclosed herein is able to adjust for changes in drinkingbehavior. For example, even if behavior changes, the algorithm stilldetects which steps have been performed, allowing general tracking ofthe position of the fluid container and a corresponding starting pointfor another sip. Consequently, even as drinking behavior changes,overall drinking behavior can be analyzed because all 8 steps are notnecessary to detect and monitor the consumption phase.

The sip detection algorithm relies on the gestures based around eightsteps of the consumption process. Of the steps listed in FIG. 15, steps4, 5, and 6 will almost always be present when a sip is being taken.Sometimes steps 3 and 4 will appear to be combined into one step as willsteps 6 and 7. Steps 1, 2, 3, 7, and 8 serve to further help detect whena sip is being taken and classify if the liquid container has been setdown, held in a hand, and other important consumption behaviors.

It is important to be able to distinctly determine step 5 because thisportion of the consumption process contains information about the user'sdrinking behavior, such as the peak x axis acceleration during the sip,the consumption duration, the rate at which the liquid container isbeing tilted, along with other information.

The information entering any of the algorithms used, including but notlimited to the sip detection algorithm, the final sip algorithm, and thecontainer and drink recognition algorithm, can be in the form of rawdata from the motion sensor(s) or could be preprocessed in any form. Anexample of this would be if the pitch angle were calculated and thenused in any respective algorithm. For example, during most consumptionprocesses, the calculated pitch angle will actually mirror the x axisaccelerations very closely, but can enhance certain movements which willmake detecting the individual steps of the consumption process moreaccurate. When the y and z axis acceleration measurements are steady,which is common for almost all consumption processes, there is littledifference between the calculated pitch angle and the x axisacceleration. However, during the last consumption process of abeverage, the y and z axis accelerations tend to move closer to 0 gthroughout Step 5. This movement dramatically increases the calculatedpitch angle, making it easier to detect the final sip of a beverage.

As explained above, the act of consuming a beverage has multiple stagesbefore the actual consumption takes place, but can also start and end ondifferent steps throughout the process. Consuming may include, but isnot limited to, any one or more of reaching, lifting, drinking, tilting,untilting, ingesting, consuming, grabbing, and/or lowering. Detectingwhen a sip is occurring before it has occurred is relatively difficult,but detecting the sip after the event has occurred becomes easier torecognize as the event has more data points to analyze. For this reason,the sip detection algorithm uses a temporary data storage that willcontinually record the accelerometer's data for a set period of time.The set period of time should be, for example, set to 10 seconds inorder to detect most consumption processes of average duration, but canbe as short or long as desired. Setting a period of time closer to 60seconds will enable detection of longer than average durationconsumption processes, such as consuming an entire beverage in onedrinking motion. Doing so will require more storage and processingcapacity for longer temporary storage. The temporary storage willcontinually be overwritten but will give the sip detection algorithmenough time to access past data to be able to recognize the drinkinggestures outlined above. FIG. 11 shows this process. Once a sip isdetected, the necessary and important information from that sip will belogged and recorded in the more permanent data storage. Relevantinformation includes but is not limited to peak x axis acceleration,Step 5 duration, duration of full consumption process, time betweensips, position of drink after consumption, full data from theconsumption phase (Step 5), and full data from the entire consumptionprocess once it has been isolated. This more permanent data storage willkeep the information on the wearable electronic device until theinformation has been sent to a phone, tablet, computer, peripheraldevice, or server. Once this has been completed, the information can bedeleted from the wearable electronic device to free up space for moredata collection.

Logging and storing the information above tracks the drinking behaviorof the user over time in addition to improving the tracking anddetection during the consumption of a single beverage. One applicationof this is that by knowing the position of the beverage (whether it hasbeen set down or is currently being held), the algorithm can adjust andlook for the consumption process to start on the appropriate step of theconsumption process. This can be seen applied to a detection algorithmin FIG. 21A. For example, if a user picks up his drink and returns it tothe table for the first sip, picks up his drink and holds it by his sidefor his second sip, and then returns the container to the table afterhis third sip, and so on, storing this information will allow theprocesser to keep tracking and analyzing the consumption of the entirebeverage over time by helping with detection.

While FIG. 11 depicts a preferred embodiment of information flow todetect and analyze a consumption process with the temporary storage, itis possible to create a detection strategy without the temporarystorage. For example, the sip detection algorithm may only measure thefirst four steps to detect a sip is about to take place before loggingthe data on the data storage during Step 5.

A few examples of sip detection algorithms are provided in FIGS. 21A,21B, and 21C. FIG. 21A shows a detection algorithm using the detectionof the eight steps of the consumption process in a linear progression inorder to determine if a sip was taken and which step to anticipate next.The benefit of this process is that it tracks if the drink has been setdown or not, and the algorithm can predict which step to look for nextwhen detecting a sip.

FIG. 21B show an algorithm that negates the need to track eachindividual step of the consumption process. A critical part of adetection algorithm is to recognize and isolate Step 5, the consumptionphase. The consumption phase will be located between Step 4 and Step 6of the consumption process. Most times, Steps 3 and 4 are combined intoone fluid movement by the user, and similarly, Steps 6 and 7 are alsocombined into one fluid movement. The algorithm in FIG. 21B looksspecifically at Steps 3 and 4, Step 5, and Steps 6 and 7 as threedistinct portions marking the preparation, the consumption, and theconclusion.

The algorithm in FIG. 21C combines the simplicity of the detectionalgorithm in FIG. 21B with also being able to track the location of thebeverage container. By having a parallel decision path looking for Step8 recognition, the algorithm can determine if the user is continuing tohold the container in a hand or if he has returned the container to aseparate resting location.

Both raw, preprocessed, and post-processed data can be used to recognizethe individual steps. Step 5 is the most significant step to be able torecognize. This step is recognized by comparing the overall movement ofa person's wrist, hand, finger, or extremity to the actual position andorientation of said wrist, hand, finger, or extremity. During Step 5,the total g is steady and nearly equal to 1, the gravitational force.This phenomena by itself means that the user is not moving the extremitywith which the sensor is located. But during Step 5, there is a steadyand resizable change in the x axis measurements. The angular pitch canalso be used instead of the raw x axis measurements. Such a change whilethe total movement is stable is very uncommon in movements other thandrinking and can be used to detect Step 5, the consumption phase.Analyzing the position of the extremity through the y and z axisaccelerations in addition to looking for the steps of the consumptionprocess before and after Step 5 increase the effectiveness of thedetection algorithm.

FIG. 18 shows different x axis accelerations throughout the consumptionof liquid in a container. The accelerations from the first sip to thelast sip will change in magnitude, but their relative characteristicsshould remain the same. Analyzing this information allows fordetermination of a consumption process ranging from a full fluidcontainer to an empty fluid container. For example, accelerations aredifferent from a full container when compared to an empty container.This difference allows for tracking how much fluid is being consumedover time. Tracking the duration of step 5 over time can also determinethe amount of fluid being consumed. Combining these approaches oftracking number of sips, duration of the consumption phase (Step 5), andother critical data increases the accuracy of determining the amount offluid consumption.

While the accelerations referenced in the above description provideenough information to recognize consumption processes, there are otherways to analyze the data in order to isolate the consumption process.For example, measuring the frequency and magnitudes of the oscillationsin the total g allow for the characterization and determination of eachdrinking step. Additionally, taking the Root Mean Square for only two ofthe axes allows for certain gesture recognition. Calculating otherinformation based on the acceleration data, such as taking a derivativeor integrating to respectively determine the change in acceleration(also known as jerk) or velocity also increases accuracy. Multiplederivatives or integrals can be calculated to determine jounce,position, or other information. A calculation for measuring therevolution around an axis was presented earlier for one axis, but thiscan be performed for all axes using the three-axis acceleration data.The addition of a three-axis gyroscope is one way to overcome potentialregions of error. Based on the data from the 3-axis accelerometer, manymore analyses can be conducted to help isolate steps of the consumptionprocess.

Last Sip Detection

Detecting the final sip of a drink is an especially critical data pointto collect in logging and tracking the amount of fluid consumed. Severalmeans for detecting the last sip are taught herein. FIG. 18, forexample, shows the x axis accelerations with a sensor on a wristthroughout the consumption of an entire beverage. Peak x axisaccelerations of consumption processes tend to rise gradually throughoutthe consumption of a beverage. This graph also shows that while the peakacceleration tends to rise, it can level off for a few sips or evendecline if the beverage is not being tilted as far back from sip-to-sip.A consistency, however, is that the last sip or those shortly before thelast sip show large differences in peak accelerations compared to theprevious sips. This phenomenon is due to the user attempting to get allof the fluid out of the container.

FIG. 12A illustrates one way of determining the final sip, by measuringonly the accelerations of the x axis. From the data collected with asensor on the wrist, the average peak x axis acceleration of a last sipacross all different fluid container types is 0.76 g with a 0.1 gstandard deviation and ranging from approximately 0.63 g to 0.95 g. Thedifference between the second-to-last and final sip averages to 0.45 gwith a 0.1 g standard deviation. Lastly the second-to-last sip averagesto 0.3 g with a standard deviation of 0.19 g and ranging from 0 g to 0.5g. Looking at the large discrepancy between the second-to-last sip andthe final sip, setting a limit or threshold on the x axis fordetermining a final sip, is an accurate means of detecting a final sip.A threshold could be, for example, 0.6 g and could range from 0.5 g to0.9 g. An issue with determining the final sip by using only a thresholdoccurs when the user will take a “second final sip.” This occurs whenthe user takes a final sip, but thinks there might be a few more dropsin the fluid container. If the user were to want to consume those finaldrops, he might take a second sip that registers a peak x axisacceleration within the “final sip” threshold.

A solution to this problem is to add a second aspect to the flowchart asseen in FIG. 12B. This aspect effectively asks the question whether theprevious sip was logged as “Drink Completed,” which will allow the finalsip algorithm to accurately log one drink for each one actuallyconsumed. There is the possibility that if the user were extremelyquenched for thirst, they might consume two full containers of liquid inonly two very large sips. If this were the case, the data for Step 5would be extremely longer than normal with a slow rise in the x axisacceleration until it reaches its peak. This anomalous consumptionprocess can be identified and marked in accordance and therefore notignored by what is seen in FIG. 12B. Once again, the drinking stepstaught herein allow for detection and analysis of many different typesof fluid consumption and behaviors, even if they change duringconsumption of one beverage or from one beverage to another.

FIG. 13 shows another possible way of determining the last sip bymeasuring the difference between the second-to-last sip and the finalsip. As stated above, the difference between the second-to-last andfinal sip averages, in this example, to 0.45 g with a 0.1 g standarddeviation. The average difference between all other sips throughout theconsumption of a drink averages to 0.04 g with a standard deviation of0.1 g and a maximum difference of 0.26 g. The smallest differencebetween the second-to-last and final sip measured was 0.35 g. Withoutany overlap between 0.26 g and 0.35 g, the final sip can be determinedby calculating the difference between sequential sips.

A third possible way of determining the last sip is by measuring thedifference between the last sip and the first sip. This is a feasibleway of logging the last sip and a completed drink, but it relies onstarting a new drink with a first sip. To improve this technique, atimer could be employed that considers the end of a drink (sip or entirecontainer) after a specified time period.

A fourth possible way of determining the last sip of liquid from acontainer is exemplified in FIG. 14. This method combines two of thepreviously disclosed methods and identifies possible last sips. If amaximum x axis acceleration is over a specified limit, such as 0.6 g,that sip will be marked as a possible last sip. Similarly, if thedifference between sips is over a threshold, such as 0.3 g, that sip canalso be marked as a possible last sip. This fourth method of determiningthe last sip differs in that a sip is only marked as “possible lastsip,” whereas the other methods definitively identify a final sip. Ifthe user starts another drink, the max x axis acceleration will be belowa threshold, such as 0.4 g, identifying the start of a new drink. Atthis point, the previous last sip will mark the end of a drink and theproper notifications and actions will take place. Similarly, if acertain time limit is reached after a sip is marked as “possible lastsip,” that sip will mark the completion of a drink. Starting anotherdrink or having sufficient time elapse would confirm that the “possiblelast sip” was indeed a last sip of the beverage container.

A fifth way of determining the last sip is by analyzing the y and z axisaccelerations during step 5. In almost all of the previous sips, the yand z axis accelerations level off and stay relatively consistentthroughout the consumption phase, step 5. The last sip, and sometimessecond-to-last sip, are the only ones that register a change in the yand z axis accelerations. They tend to decrease as the x axisacceleration is continuing to increase. This is attributed to a changein the user's elbow position during consumption in an attempt to tip theliquid container in a more vertical position. The final sip algorithmcould be set to log a sip as a last sip if the y and/or z axisaccelerations decrease by more than 0.2 g throughout step 5. Thisalgorithm is shown in FIG. 22.

The methods above describe different approaches of determining the lastsip based on data from an accelerometer on a wrist and correspondinglimits and thresholds. These methods can be adapted for use with anaccelerometer placed on the actual beverage container in order todetermine when a drink or container of liquid has been completelyconsumed. Different limits and thresholds would need to be used based oncorresponding data for the technique, but would follow the reasoningexplained herein. For example, when the sensor is on the fluidcontainer, primarily the x axis accelerations matter. With this sensorlocation, the method of determining a last sip shown in FIG. 13 is arelevant algorithm to use. The limit used for an on-wrist sensor is 0.3g and this limit will also work well for an on-drink sensor location. Ifthe method of determining a last sip shown in FIG. 12A or 12B were to beused, the limit would have to be decreased to 0.2 g from 0.6 g shown inFIG. 12A or 12B. The problem with this method is that overlap existsdepending on what type of fluid container is used. A solution to thispotential problem is a more complex algorithm, like that shown in FIG.14, which should be used with the new on-drink sensor limit of 0.2 g.

If a user is drinking from multiple containers, for example, both water(from a glass) and coffee (from a mug) with consumption processesoverlapping in time period, say during the same meal, one way of solvingthis particular quandary vis-à-vis the invention taught herein is asfollows. In most situations similar to this, one of the multiple drinksconsumed is likely to be water. For this reason, an example of asolution is to have a button (or some other means of signification) onthe wearable electronic device which can be used to log when a sip iswater and when it is not. This way, the wearable electronic device willstill be able to track the consumption of the other non-water drink,while also keeping a better log of the user's water intake. This methodcan apply to any situation in which there is a primary drink and asecondary drink. In the previous example, water was the secondary drink,but the user decides which drink is primary and which drink issecondary.

With a system similar to this, data about each sip and consumptionprocess will be stored, and the sips from a secondary drink (e.g.,water, soda, etc.) will be marked or flagged as secondary. This willallow certain algorithms using this data to ignore the secondary sipsand focus on the primary sips. An example of this is with the containerand drink recognition algorithm. If this algorithm is processingnecessary information (e.g., peak x axis accelerations, pitch angle,etc.) from a series of sips in which some have been from a secondarydrink, the algorithm will be able to remove the sips marked secondary,and focus on the sips marked primary to determine the type of containerthe beverage is being consumed from or the actual beverage itself.Similarly, a system like this can be used for the final sip detectionalgorithm, properly knowing when the primary or secondary drink has beencompleted.

Conversely, if this algorithm is processing necessary information peak xaxis accelerations, pitch angle, etc.) from a series of sips in whichsome have been from a secondary drink, the algorithm will be able toremove the sips marked primary, and focus on the sips marked secondaryto determine the type of container the secondary beverage is beingconsumed from or the actual beverage itself. While the example in thiscase uses one button, the device itself could have multiple buttons tomark a second beverage, third beverage, fourth beverage, and so on, ifthe user decides to have multiple beverages.

Sometimes drinks are being constantly refilled. Both water and coffeeare examples of this phenomenon when consuming these beverages at arestaurant, for example. In these situations, when a last sip cannot bedetected, a difference in peak x axis accelerations can detect andregister a refill. Consequently, total amount consumed may be calculatedby the duration of Step 5 from the data and other learned averages ofthe user (i.e., how much liquid the user typically consumes in one sip).

In interpreting and analyzing the raw or preprocessed data in any of thealgorithms used, the magnitude of data, duration of events, frequency ofevents, time or distance between events, and other characteristics fromthe motion sensors are critical in determining necessary information,including but not limited to if the consumption process has occurred,the amount of liquid consumed, the type of liquid container, and thetype of liquid being consumed.

Container Recognition

Recognizing the type of container from which the liquid is beingconsumed (e.g., open-top glass, mug, bottle, aluminum can, stemmed-wineglass, etc.) helps to also recognize the type of liquid being consumed.Whereas most diet/fitness trackers require the user to input when theuser consumed a beverage, how much was consumed, and the type ofbeverage—and U.S. patent application 2014/0372045A1 requires the user toenter the size and type of container the liquid is contained in—thecurrent invention can determine not only when a liquid is beingconsumed, but also the type of container from which it is beingconsumed.

This method of container recognition is based on scientific researchfinding that each type of glass or other type of container has a uniquepattern when plotting and/or regressing the peak x axis acceleration oneither the time or number of sips taken throughout the consumption of abeverage. In FIG. 18, for example, the peak x axis acceleration for eachindividual sip is shown to be gradually rising throughout theconsumption of the beverage when looking at the overall trend. FIG. 18shows this trend when the sensor is placed on the wrist of a user, but asimilar trend is also present when the sensor is on the drink. The lastsip has a significantly higher peak x axis acceleration than the priorsips. The peak x axis acceleration is found during Step 5 of the eightsteps of drinking seen in FIG. 15 and corresponding FIG. 17. The firstsip peak x axis acceleration, last sip peak x axis acceleration, and allpeak x axis accelerations from sips between the first and last sips haveunique values based on the type of container from which the liquid isbeing consumed. Data from an experiment is plotted in FIG. 20 showingthe peak acceleration for each sip throughout the consumption of a drink(excluding the last sip) for different types of fluid containers with anon-wrist sensor location. Similar trends can be seen when the sensor isplaced on the drink itself when regressing the x axis peak accelerationson time or the number of sips.

FIG. 19 shows peak x axis accelerations from a sensor placed on thedrink plotted against the number of sips throughout the consumption of abeverage, taken from six unique and common types of beverage containers.If using this method to recognize the type of container the liquid isbeing consumed from, it is a preferred embodiment to factor in sips S₁to S_(n-1). Because the last sip S_(n) usually has a much higher peakacceleration, it skews the results, affecting the ability to mostaccurately categorize the type of beverage container from which theliquid is being consumed through a linear regression. Other types ofanalysis are capable of determining more complex trends that can allowthe last sip (S_(n)) to be included in said analysis. A linearregression is also depicted between the peak accelerations and thenumber of sips from each container type. The results of the regressionare shown as a trend line on the chart in addition to the equation.Looking at the six different containers, each type has a unique startingpoint, unique ending point, and unique slope. For example, fully filledopen top containers such as a pint glass or coffee mug havesignificantly lower first sip peak accelerations when compared to acontainer with a constrained opening such as a glass bottle or aluminumcan. A partially filled open top container such as a wine glass ortumbler glass containing liquor tends to demonstrate first sip peakaccelerations between a fully filled open top container and a containerwith a strained opening such as a glass bottle.

FIG. 20 shows similar data to that of FIG. 19, but with an on-wristsensor location instead of a sensor placed on the beverage container.The containers on both charts include a 12 oz. glass bottle, a 16 oz.pint glass, a 12 oz. aluminum can, a tumbler or low ball glasscontaining 1.5 oz, of liquor, a 12 oz. coffee mug, and a standardstemmed wine glass containing 6 oz. of wine, although this method ofdetermining container type can be applied to any liquid container.

By comparing the real-time data being collected by the wearableelectronic device to the known trends of different types of containers,the invention can determine the type of container from which a liquid isbeing consumed. The regression analysis determines the type ofcontainer, but it is not necessary since other techniques exist asdescribed herein. For example, knowing the average difference of peakaccelerations between each sip and the peak acceleration of the firstsip is enough to reasonably classify the type of container being used.Through machine learning, the wearable electronic device and/or programon a phone, tablet, computer, other peripheral device, or server canlearn the habits of each user and provide more accurate results overtime based on the user's inputs and selection/correction of fluidcontainer types.

Based on knowing the type and size of container, the wearable electronicdevice or the program on the phone, tablet, other peripheral device, orcomputer will be able to recognize or recommend the type of drink beingconsumed, making it easier for the user to actually log the drinkconsumed. It is also possible to narrow the selection of possible drinksbeing consumed in a similar manner to what is described for thecontainer type. For example, a hot tea or hot coffee will be consumeddifferently from a mug than if room temperature water were consumed fromthat same mug. Through machine learning, it is possible to detect thetype of drink by knowing the container type and how the user tends toconsume different types of beverages. For example, a user could drinkeither hot tea or coffee from a typical coffee mug. After loggingmultiple events during which he drank hot tea and coffee from a similarmug, the program can learn slight differences in the way they areconsumed. One way this could be performed is by detecting and analyzingnumber of sips and/or duration of sips. The duration of each sip of hottea might be longer and more gradual resulting in a reduced slope of thex axis acceleration during consumption (Step 5) when compared to coffee.From the container recognition, the program will know what type of fluidcontainer the beverage is being consumed from. Based on suchinformation, the program can narrow the possible drinks to a limitednumber of possibilities. After collecting data from the specific user,the program will learn how the user drinks each type of beveragedifferently. The program can assign the type of drink automatically,negating the need for the user to enter the type of drink they areconsuming. To ensure accuracy and help with learning, the program couldgive the user the option to edit or change the type of drink consumedafter the event has occurred.

Knowing the type of drink is necessary when applying nutritionalinformation to the consumption of any fluid. Either through user inputor through the container and drink recognition algorithm describedabove, each sip and or complete drink will be marked with the type offluid the user consumed. Through either information stored on a remoteserver in the cloud, through information stored on the device, orinformation stored on a peripheral electronic device, each sip and/orcompleted beverage can be marked with the type of drink and nutritionalinformation. By knowing the type of drink, the nutritional informationrelating to that type of drink, and the quantity consumed, the user willbe able to look up nutritional information of their complete fluidconsumption over any period of time.

In any of the algorithms used, there are many tools to help with theanalysis of the raw, preprocessed, or post processed data. For example,the descriptions above specifically mention running a linear regressionon the peak accelerations from sips throughout the entire beverage inorder to determine the type of container. The description for the finalsip detection mentions multiple ways in determining when the final siphas occurred by looking at the magnitude of the peak acceleration,difference between sequential peaks, and other methods. Some otheruseful tools for any of the algorithms include, but are not limited to,running parametric models (e.g., Autoregressive models), FourierTransforms, Fast Fourier Transforms, Stochastic estimations, LinearPredictive Analysis, and other filtering techniques (e.g., Least MeanSquares, Kalaman, etc.).

With any of the algorithms used, applying machine learning can result inmuch more accurate prediction and detection of desired information(e.g., the consumption process, the final sip, the type of container,the type of beverage, etc.). When the user is active in marking when asip is taken, for instance from a secondary drink or when the user marksthe exact type of beverage consumed, the data gathered can be used astraining data to improve the algorithm through supervised learningtechniques such as Random forests. In other situations, machine learningcan be used to better understand the individual user, and adapt thealgorithms to that user's specific habits and movements by looking atpatterns over time.

Other substantially and specifically practical and useful embodimentsmay become apparent to those skilled in this art from reading theabove-recited and/or herein-included detailed description and/ordrawings of certain exemplary embodiments. It should be understood thatnumerous variations, modifications, and additional embodiments arepossible, and accordingly, all such variations, modifications, andembodiments are to be regarded as being within the scope of thisapplication.

The invention claimed is:
 1. A method for detecting when a user isconsuming fluid during an individual sip, comprising: providing anelectronic device capable of being worn on an extremity of a user,detecting movement, position, or orientation of said electronic deviceand said extremity by way of a motion sensor provided by said electronicdevice, receiving and analyzing signals from said motion sensor by wayof a microprocessor capable of running a programmed algorithm, detectingwhen the user performs one or more actions that indicate the user isconsuming a fluid during an individual sip by isolating and marking astart of actual fluid consumption when a tilting action is recognizedand marking an end of actual fluid consumption when an untilting actionis recognized, using said programmed algorithm, wherein the tiltingand/or untilting action is recognized by comparing a standard deviationof a total acceleration (total g) of the electronic device for a setperiod of time to individual accelerations of one or more x-, y-, orz-axes, position, change in the x-axis acceleration, or orientation ofsaid user's extremity and/or said electronic device and wherein thetotal acceleration is determined by: total g=√{square root over(x²+y²+z²)} where x, y, and z are the individual accelerations of thex-, y-, and z-axes, respectively, wherein the tilting action isrecognized by the following criteria: a) a root mean square of they-axis acceleration and the z-axis acceleration is greater than 0.5 gand less than 1.4 g; b) the total acceleration is greater than 0.8 g andless than 1.2 g and/or the standard deviation of the total accelerationis less than 0.05 g within a window equal to or less than 1 second; andc) a change in the x-axis acceleration is greater than 0 g and less than1.1 g every 1 second.
 2. The method of claim 1, wherein said programmedalgorithm is capable of recognizing specific gestures of said extremityand filtering out non-drinking movements in determining whether saiduser is consuming a fluid.
 3. The method of claim 1, wherein saidprogrammed algorithm is capable of isolating when fluid is beingconsumed by the user and analyzing information from this consumptionperiod to determine certain drinking behaviors.
 4. The method of claim1, wherein a change in position or orientation is recognized by way ofat least one gyroscope and/or at least one accelerometer provided bysaid electronic device.
 5. The method of claim 1, wherein said motionsensor comprises a gyroscope and/or accelerometer and the one or moreactions are recognized by analyzing specific movements, orientations orpositions of said user's extremity and said electronic device.
 6. Themethod of claim 1, wherein said programmed algorithm is capable ofcomparing overall movement of said user's extremity and said electronicdevice to changes in position or orientation of said extremity and saidelectronic device.
 7. The method of claim 1, wherein a change inposition or orientation of said extremity and said electronic device isdetermined by monitoring pitch angle or rotation about an axis parallelto said user's extremity which is a forearm.
 8. The method of claim 1,wherein a change in position or orientation of said extremity and saidelectronic device is determined by monitoring x, y, or z axisaccelerations and total accelerations of said extremity and saidelectronic device.
 9. The method of claim 1, wherein one or more of thefollowing gestures are detected to indicate whether said user isdrinking: i) Reaching for drink; ii) Grabbing drink; iii) Lifting drinkto mouth; iv) Tilting until liquid touches lips; v) Tilting whileconsuming; vi) Untilting by reversing the drinking tilt to normalholding position; vii) Lowering drink; and viii) Returning drink andarm.
 10. The method of claim 1, wherein said microprocessor producesinformation that is provided to said user as results of said analyzingor a summary thereof.
 11. The method of claim 1, wherein saidmicroprocessor produces information that is provided to said user topresent consequences of drinking behavior of said user based on metricsrelated to physical performance, as well as physical and/or mentalhealth issues impacted by drinking behavior.
 12. The method of claim 1,wherein detecting when said user is consuming a fluid is based on astandard deviation of the total acceleration of below 0.15 g within awindow equal to or less than 1 second.
 13. The method of claim 8,wherein the root mean square of the y and z axes are used to determinewhether said user has lifted said extremity to a drinking position nearthe user's mouth.
 14. The method of claim 1, further comprisingestimating the amount of fluid consumed during the individual sip basedon the magnitude of measured x-axis accelerations, allowing for peakx-axis acceleration measurements ranging from −0.8 g to 1 g.
 15. Themethod of claim 1, further comprising estimating the amount of fluidconsumed during the individual sip based on a duration of the individualsip and a rate of tilt in the x-axis.
 16. A method for detecting when auser is consuming a fluid during an individual sip and estimating anamount of fluid being consumed, comprising: providing an electronicdevice capable of being worn on an extremity of a user, detectingmovement, position, or orientation of said electronic device and saidextremity by way of a motion sensor provided by said electronic device,receiving and analyzing signals from said motion sensor by way of amicroprocessor capable of running a programmed algorithm, detectingwhether the user performs one or more actions that indicate the user ispreparing to take an individual sip, using said programmed algorithm,wherein the following gestures are detected to indicate the user ispreparing to take an individual sip: i) Lifting drink to mouth; ii)Tilting until liquid touches lips; detecting and marking when fluidconsumption has begun, using said programmed algorithm to recognize thefollowing gestures: iii) Tilting while consuming, which is recognized bythe following criteria: a) detecting continuous increases in a measuredx-axis acceleration and allowing for measured x-axis accelerationsgreater than −0.8 g and less than 1 g; b) a root mean square of a y-axisacceleration and a z-axis acceleration is greater than 0.5 g and lessthan 1.4 g; c) a measured total acceleration is greater than 0.8 g andless than 1.2 g and/or a measured total acceleration standard deviationis less than 0.05 g within a window equal to or less than 1 second; andd) a change in the measured x-axis acceleration is greater than 0 g andless than 1.1 g every 1 second; detecting and marking an end of fluidconsumption, using said programmed algorithm to recognize one or more ofthe following gestures: iv) Untilting by reversing the drinking tilt tonormal holding position; v) Lowering drink, and calculating the estimateof an amount of fluid consumed during the individual sip by measuring amagnitude of the x-axis accelerations during the tilting while consuminggesture and/or by measuring a duration of the tilting while consuminggesture.
 17. The method of claim 16, wherein the untilting action(gesture iv.) is recognized by the following criteria: a. A change inthe x axis accelerations are less than −0.3 g every 1 second; and b. Ameasured total acceleration is greater than 0.7 g and less than 1.3 gand/or the measured total acceleration standard deviation is less than0.5 g.
 18. The method of claim 16, wherein the lowering drink action(gesture v.) is recognized by the following criteria: A minimum measuredx axis acceleration within a set period reaches a value below −0.5 g.19. The method of claim 16, wherein calculating the estimate of theamount of fluid consumed during the individual sip by measuring themagnitude of the x axis accelerations during the tilting while consuminggesture is performed using an average x axis acceleration.
 20. Themethod of claim 16, wherein the amount of the fluid consumed during theindividual sip is further calculated by a total change in magnitude ofthe x axis acceleration during the tilting while consuming gesture. 21.The method of claim 16, wherein the amount of the fluid consumed duringthe individual sip is further calculated by a rate of change in the xaxis acceleration during the tilting while consuming gesture.
 22. Themethod of claim 16, wherein the amount of fluid consumed during theindividual sip is further determined by using the sip duration tocalculate an average rate of change in the x axis acceleration duringthe tilting while consuming gesture.
 23. The method of claim 16, whereinthe lifting drink to mouth action (gesture i.) is recognized by thefollowing criteria: a. A minimum x axis acceleration is less than −0.5 gat the beginning of gesture i.; b. A measured total acceleration isgreater than 0.7 g and less than 1.3 g and/or the measured totalacceleration standard deviation is less than 0.5 g; c. A change in the xaxis acceleration is greater than 0.1 g every 1 second; and d. A changein the absolute value of the y axis acceleration is greater than 0.04 gevery 1 second.
 24. The method of claim 16, wherein the tilting untilliquid touches lips gesture (gesture ii) is recognized by the followingcriteria: a. A measured total acceleration is greater than 0.7 g andless than 1.3 g and/or the measured total acceleration standarddeviation is less than 0.5 g; b. A change in the x axis acceleration isgreater than 0.1 g every 1 second; and c. A change in the absolute valueof the y axis acceleration is less than 0.04 g every 1 second.
 25. Themethod of claim 24, wherein the minimum x axis acceleration over a timeperiod of no longer than 10 seconds prior to fluid consumption is usedto determine if a lifting or tilting event has occurred prior to aconsumption phase.
 26. An apparatus comprising: A device capable ofbeing worn on an extremity by a user, wherein the device comprises amotion sensor, microprocessor, and programmed algorithm, and wherein thedevice is capable of being programmed to detect when the user is takingan individual sip, and to detect an amount of fluid consumed,comprising: detecting movement, position, or orientation of the deviceand said extremity by way of said motion sensor provided by said device,receiving and analyzing signals from said motion sensor by way of saidmicroprocessor capable of running said programmed algorithm, anddetecting when the user performs one or more actions that indicate theuser is consuming a fluid by determining and marking when the userstarts consuming the fluid during the individual sip and when the userstops consuming the fluid during the individual sip, using saidprogrammed algorithm, wherein said one or more actions are recognized bythe following criteria: a) comparing total acceleration (total g) of thedevice to individual accelerations of one or more x-, y-, or z-axes,position, or orientation of said user's extremity and/or said device andwherein the total acceleration is determined by: total g=√{square rootover (x²+y²+z²)} where x, y, and z are the individual accelerations ofthe x-, y-, and z-axes, respectively; b) a root mean square of a y-axisacceleration and a z-axis acceleration is greater than 0.5 g and lessthan 1.4 g; c) the total acceleration is greater than 0.8 g and lessthan 1.2 g and/or a total acceleration standard deviation is less than0.05 g within a window equal to or less than 1 second; and d) a changein an x-axis acceleration is greater than 0 g and less than 1.1 g every1 second; wherein the amount of fluid being consumed is calculated basedon a magnitude of the x-axis accelerations between the marked start ofthe fluid consumption and the marked end of the fluid consumption,wherein a sip is capable of being detected between −0.8 g and 1 s.